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ABSTRACT 



Analyses were conducted to determine whether the two 
statistical tests of significance are identical. Such demonstrations 
regarding parallel correlational and analysis of variance procedures have 
brought further understanding to each of the domains in the past. When a 
researcher is investigating the relationship between two variables and thinks 
that the relationship may be different for two subpopulations under 
consideration, that question can be tested. Some texts present this test as 
the test of significance for the difference between two correlations. The 
test uses the Fischer "z" transformation on the two sample correlations. The 
interaction between a dichotomous variable and a continuous variable is often 
of interest and can be tested within the general linear model approach. Two 
examples in which the z and F results are the same, and two in which they are 
different are presented. For most practical purposes, the two conceptually 
similar tests of significance actually test slightly different questions. The 
research hypothesis should be stated clearly to assure use of the right test 
of significance. (Contains 2 tables, 4 figures, and 16 references.) (SLD) 
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Purpose 

The purpose of this investigation was to determine when the two statistical tests 
of significance are identical. Such demonstrations regarding parallel correlational and 
analysis of variance procedures have, in the past, brought further understanding to 
each of the domains (McClendon, 1994; McNeil & Beggs, 1969; McNeil, Newman, & 
Kelly, 1996; Ward & Jennings, 1973; Williams, 1974). 



Inspiration for the Investigation 



Several years ago the second author intimated, during a presentation at the 
American Educational Research Association, that the two tests of significance were the 
same (Newman, personal communication). While this made sense to the first author, 
investigation of numerous statistical texts has resulted in not discovering any such 
demonstration. 
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Difference Between Two Correlations 

When a researcher is investigating the relationship between two variables, and 
suspects that the relationship may be different for two subpopulations of the population 
under consideration, that question can be tested. Some statistical texts present this 
test as the test of significance for the difference between two correlations, although not 
all texts discuss this test (e.g., Gravetter & Wallnaw, 1992; Pagano, 1994; McClave & 
Benson, 1998). The test uses the Fischer z transformation on the two sample 
correlations as the sampling distribution for population correlations (other than a 
population correlation of 0) will not approach normality. The test only depends on the 
sample correlations and the size of the samples. The test is a z test as follows: 

( z rl - z r2 ) - (the hypothesized difference between the two correlations) 
z = 



square root of 1 / [ (n i - 3 ) + 1 / (n 2 * 3) ] 

Three degrees of freedom are lost for each sample because the Fischer z 
transformation is used. The hypothesized difference between the two correlations will 
often be zero, as in all of the examples in this paper. The hypothesized difference 
between the two correlations would not have to be zero, and that would be a natural 
extension of this investigation (see McNeil, 1991 for non-zero restrictions on other 
statistical tests). This test is presented in such tests as Hinkle, Wiersma, and Jurs 
(1994, p. 263). Figure 1 is a plot of the data used for case 1 . A “1” indicates data from 
group 1 and a “2” indicates data from group 2. Analysis of the data in Figure 1 is the 
limiting case when the slopes are the same and the standard deviations with the 
groups are the same. Other analyses consider same slopes with different standard 
deviations, and different correlations with same slopes. 

Interaction Between a Dichotomous Variable and a Continuous Variable 

While most statistics texts provide the impression that interaction is only of 
interest in guiding the researcher to either simple effects or main effects (such as in a 



two-way design), interaction may be of interest in its own right. For example, the area 
of aptitude by treatment interaction, and matching the instructional style with the 
learning style are two such areas of investigation wherein interaction is of primary 
importance. Furthermore, the discovery of interaction may help clear up conflicting 
results in a given area (Bender, Kelly, & Pierson, 1968; McNeil & Newman, 1995; 
McNeil, Newman, & Kelly, 1996). 

Interaction can probably best be understood by reference to Figures 1 and 2. 
Figure 1 depicts the state of affairs wherein there is no interaction. Figure 2 depicts the 
interaction between the dichotomous variable (say, teaching style) and the continuous 
variable of X (say, aptitude). The suspected interaction can be tested within the 
General Linear Model approach (GLM) by developing a Full Model which allows two 
lines of best fit (one for group 1 and one for group 2): 

Full Model (Two lines): Y = a-| Ui + bi Xi + a 2 U 2 + b 2 X 2 + Ei 

where: Ui = 1 for participants in group 1, 0 otherwise, 

Xi = the continuous value of X for participants in group 1, 0 otherwise, 

U 2 = 1 for participants in group 2, 0 otherwise, 

X 2 = the continuous value of X for participants in group 2, 0 otherwise, 
and ai t bi, a2, and b2 are least squares weighting coefficients 

calculated so as to minimize the sum of the squared errors in each of the 
error vectors, Ei and E 2 . (a-| and a2 are the two intercepts and bi and 

b2 are the two slopes.) 

In order to test for interaction, one must see how the model of two parallel lines fits the 
data. Parallel lines means that the slopes of the two lines are the same. Therefore 
one must restrict the two slopes in the above “two line model” to be equal to a common 
slope, say “b.” This results in the following Restricted Model: 

Restricted Model (Two parallel lines): Y = ai Ui + b Xi + a2 U 2 + b X 2 + E 2 
but [ b Xi + b X 2 ] = b X, and therefore the above model is equivalent to: 
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Restricted Model (Two parallel lines): Y = a-| Ui + b X + a2 U2 + E2 
where b is the common slope for each of the two groups. 

This test of interaction is presented in various GLM texts (Kerlinger & Pedhazur, 
1973; McClendon, 1994; McNeil, Newman, & Kelly, 1996; Ward & Jennings, 1973). 
Since there are 4 predictor variables in the Full Model and 3 in the Restricted Model, 
there is 1 (4 - 3) degree of freedom in the numerator of the F test. The denominator 
degrees of freedom are the total number of participants minus the 4 pieces of 

information in the Full Model, or n 1 + n 2-4. Since z 2 = F when there is one degree 

of freedom in the numerator, and an infinite degrees of freedom in the numerator, the 
two tests could produce the same answer under the condition of an infinite degrees of 
freedom in the denominator--when there is a very large number of participants in the 
two samples. The astute reader will realize that the degrees of freedom in the 
numerator for the two tests are indeed 1, but the degrees of freedom denominator are 
different, thus the two tests will usually not be exactly the same. 

Case 1 

In case 1 , group 2 data was an exact replication of group 1 data (see Table 1). 
The means, standard deviations, and correlations within the two groups were exactly 
the same. The slopes were also exactly the same. Since there was no difference 
between the correlations, the z test was also equal to 0.00. Hence for case 1 , the two 
tests of significance were identical. 

Case 2 

By adding a constant of 7 to all the X scores in group 1 , the data for group 1 is 
moved 7 units to the right, as in Figure 1 . The correlations are not affected (still .57) by 
this linear transformation, and the slopes are also not affected (still 1.00). The two 
tests of significance will each yield a value of 0.00, as indicated in Table 1 for case 2. 

Case 3 

Case 3 considers data that has an opposite correlation within the two groups, 
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as in Figure 2. As Table 1 indicates, the two tests of significance were quite 
discrepant for case 3. 

Case 4 

Case 4 considered the possibility of different standard deviations in the two 
groups. The data from case 2 was modified by making the deviations from the line of 
best fit within group 1 only one unit, rather than 2 units as in group 2 (as indicated in 
Figure 3). Clearly, the slopes are the same within each group, resulting in an F of 
0.00, but the correlations are different (.81 vs. .57), resulting in a z value of .87. 
Conclusion 

The two tests of significance are isomorphic only when within the two groups 
there is the same correlation, the same variance on X, and the same variance on Y. 
Except in the limiting cases, the two tests are testing different questions. If the question 
is about slopes, irrespective of variances, the interaction test is appropriate. If the 
variances within groups are of concern, then the correlation test is appropriate. One 
should note that a correlation coefficient is independent of means and variances. 

Thus the two groups, while allowed to have these differences, are in a sense forced to 
have the same means and variances when the correlation test of significance is used. 
The interaction test of significance analyses the data the way it is, not forcing the two 
groups to have the same means and variances. Thus, for most practical purposes the 
two tests of significance, while conceptually similar, actually test slightly different 
questions. Therefore, one test is not better than the other, they simply test similar, but 
different questions (except in the trivial situation of case 1). If one does not clearly 
state the research hypothesis, one might use the wrong test-resulting in a Type VI 
error as discussed by Newman and Newman (1994). 

Epilogue I 

In discussing the rough draft of the present paper, we decided to investigate 
one additional case, wherein the data was standardized within each of the two 
groups, discussed below as case 5. 
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Case 5 



Case 5 considers a data set (case 4 data) that has been standardized within 
each group. The group mean was subtracted from each X and Y score, and then 
divided by the appropriate standard deviation (creating z scores within each group). 
The correlations within the groups remain the same because this is a linear 
transformation, but the slopes are now different. Instead of slopes of 1.00 within each 
of the two groups, the slopes are now equal to the correlations (.81 and .57 
respectively). The test for the difference between the two correlations was still a z of 
.87. The F test for linear interaction was .51 . We had expected the two results to be 
the same (that is z*z= F ). 

Going back to Fisher (1941), it is claimed that the Fisher z transformation is 
approximately normal, and is appropriate for moderately sized samples. Therefore we 
investigated larger sample sizes. Table 2 contains the results for N of 20, 40, 80, 160, 
and 320 per group. Note that as the sample size becomes larger, that z *z is more 
and more closely equal to twice that of the F test, though we had expected the two to 
become similar. 



Epilogue II 

In discusssing these results amonst ourselves, we have concluded that the 
reason that we are getting different results is indeed because different questions are 
being asked. The interaction question is actually testing the partial correlations, 
whereas the Fisher z test is testing the correlations. Similar questions, but different 
answers. 
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Figure 1 . No interaction, with data in group 1 7 units higher on X than data in group 2. 
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Figure 2. Interaction between group and X. 
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Figure 3. No interaction (parallel slopes), but less error variance in group 1, and hence 
higher correlation between X and Y in group 1 than in group 2. 
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OPTIONS L S = 7 0 ; 

TITLE DIFFERENCE BETWEEN TWO CORRELATIONS- -SIGNIFICANT INTERACTION; 
TITLE2; 

TITLES ONE GROUP OFFSET 7 POINTS; 

TITLES DIFFERENT CORRS, SAME SLOPE; 

OPTIONS LINESIZE = 72;. 

DATA NEW; 

INPUT X Y GROUP; 

G1 = 0; G2 = 0; XSL1 = 0; XSL2 = 0; 

IF GROUP = 1 THEN G1 = 1 ; IF GROUP = 1 THEN X = X+7 ; 

IF GROUP = 2 THEN G2 = 1 ; 

SL1 = X * Gl; 

S L 2 = X * G2; 

CARDS; 



1 2 1 

1 A 1 

2 3 1 

2 5 1 

3 A 1 
3 6 1 
A 5 1 
A 7 1 
5 6 1 
5 0 1 
1 0 2 
1 A 2 



2 1 2 

2 5 2 

3 2 2 
3 6 2 
A 3 2 
A 7 2 
5 A 2 
5 P 2 

FROC SORT ; BY GROUP; 

PFOC CORR; BY GROUP; 

FFCC REG; MODEL Y = SL1 SL2 Gl ; 
TEST SL1 = 3L2 ; 

PROC PLOT; PLOT Y * X =GROUP; 



Figure 4. SAS setup for data in Figure 3. 
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Table 1 

Results from the Various Analyses 



Case 






GrouD 1 






Grouo 2 




Statistics 


M x 


My 


Sx 


Sy 


r 


M x 


My 


Sx 


Sy 


r 


z F 


1 


3 


4 


1.49 


1.49 


.57 


3 


4 


1.49 


1.49 


.57 


0.00 0.00 


2 


10 


4 


1.49 


2.58 


.57 


3 


4 


1.49 


2.58 


.57 


0.00 0.00 


3 


3 


5 


1.49 


1.82 


.81 


3 


5 


1.49 


1.82 


-.81 


4.21 32.00 


4 


10 


5 


1.49 


1.82 


.81 


3 


4 


1.49 


2.58 


.57 


.87 0.00 



Note. N in both groups is always 10. 
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Table 2 

Results from Different Sample Sizes 



Sample Size per Group 


Z 


z*z 


F 


10 


.87 


.75 


.51 


20 


1.73 


2.97 


1.03 


40 


2.02 


4.07 


2.17 


80 


2.91 


8.46 


4.46 


160 


4.34 


18.84 


9.04 


320 


6.09 


37.13 


18.22 
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